Natural Language Engineering
نویسندگان
چکیده
In this paper, we study the problem of structural analysis of Web documents aiming at extracting the sectional hierarchy of a document. In general, a document can be represented as a hierarchy of sections and subsections with corresponding headings and subheadings. We developed two machine learning models: heading extraction model and hierarchy extraction model. Heading extraction was formulated as a classification problem whereas a tree-based learning approach was employed in hierarchy extraction. For this purpose, we developed an incremental learning algorithm based on support vector machines and perceptrons. The models were evaluated in detail with respect to the performance of the heading and hierarchy extraction tasks. For comparison, a baseline rule-based approach was used that relies on heuristics and HTML document object model tree processing. The machine learning approach, which is a fully automatic approach, outperformed the rule-based approach. We also analyzed the effect of document structuring on automatic summarization in the context of Web search. The results of the task-based evaluation on TREC queries showed that structured summaries are superior to unstructured summaries both in terms of accuracy and user ratings, and enable the users to determine the relevancy of search results more accurately than search engine snippets.
منابع مشابه
An approach to program understanding by natural language understanding
Published in Natural Language Engineering, Cambridge University Press, Vol. 5, Issue 1, pp.1-18, 1999 (Copyright Cambridge University Press, 1999) An Approach to Program Understanding by Natural Language Understanding Letha H. Etzkorn, Lisa L. Bowen, Carl G. Davis Computer Science Department The University of Alabama in Huntsville Huntsville, AL 35899 [email protected], [email protected], cda...
متن کاملCurrent Issues In Software Engineering For Natural Language Processing
In Natural Language Processing (NLP), research results from software engineering and software technology have often been neglected. This paper describes some factors that add complexity to the task of engineering reusable NLP systems (beyond conventional software systems). Current work in the area of design patterns and composition languages is described and claimed relevant for natural languag...
متن کاملThe role of natural language in requirements engineering
It is argued that the potential role of natural language processing in the requirements engineering process has been overstated in the past, possibly due to fundamental misunderstandings of the requirements engineering process itself. Since more realistic ambitions are likely to lead to less disappointment in the future, an effort is made to identify some phases and tasks where natural language...
متن کاملNatural Language User Interface For Software Engineering Tasks
In this paper, we present the idea to use natural language as the user interface for programming tasks. Programming languages assist with repetitive tasks that involve the use of conditionals, loops and statements. This is what is often challenging users. However, users can easily describe tasks in their natural language. We aim to develop a Natural Language User Interface that enables users to...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2015